Shiny for R-Package Risk Assessment


Aaron Clark, 16 Aug 2023

Disclaimer




Any opinions expressed in this presentation and on the following slides are solely those of the presenter and do not necessarily reflect those sponsoring the work

  • Consortia of ~50 companies (mostly pharma and biotech)

  • Focus on designing/building a framework to validate R and R packages with an eye to all open-source software.

  • The main goal is ensuring proper validation documentation exists in accordance with expectations of regulatory agencies.

Checkout pharmar.org to learn about all work streams and get involved!

Two tools: what do they do



is a framework to quantify an R package’s “risk” by assessing several meaningful metrics designed to evaluate package development best practices, code documentation, community engagement, and development sustainability.


is a full-fledged R package containing a shiny front-end that augments the utility of {riskmetric}. The application’s goal is to provide a central hub for an organization to review and assess the risk of R packages, providing handy tools and guide rails along the way.

Quantify risk programmatically

Sometimes “quality” is measurable! Software dev best practices dictate an R-package should have:

  • A license
  • Source code available for browsing
  • An easy to contact maintainer
  • A place to report bugs
  • Evidence that new bugs are being addressed
  • Complete Function documentation
  • Adequate test coverage
  • Community usage

18 total assessments (to date)!

Why create a Shiny app?

Our audience / users: those those making ‘package inclusion’ requests for GxP environments. They need support to…

–

  • Generate consistent, org-specific analysis of risk outputs & a means for package exploration without the need to write R code

–

  • Run {riskmetric} on the same machine with the same environment – creating a central hub for reproducibility

–

  • Automate ‘decision triage’ based on pre-defined rules, saving time & mental energy

–

  • Manage who’s involved in the review process with user authentication with user role management

–

  • Facilitate and store communication between many users, on certain packages and/or metrics

–

  • Generate risk summary reports, with opportunity to inject opinions, for sharing with the decision making parties

Why create a Shiny app?

Highest and best use of the app? It’s all about two things:

  • forcing org members to take responsibility for assessing package risk themselves prior to making an IT request
  • generating a summary report for IT that shows the requestor has done their due diligence meeting inclusion requirements

[pic of example report and arrow to ppm]

An exercise: Review the XYZ package


Shinyapps.io

Latest features of v2.0.0

Most notably:

  • Face lift to the 'Report Builder' & 'Database View'
  • Better support to analyze dependencies
  • Added unparalleled customization, including the use of a configuration file
  • Allow admin users to edit roles and privileges on the fly
  • Allow users to explore source contents of package

The feedback loop is crucial! Submit an issue on GitHub today.

Latest: 'Report Builder' Face Lift

  • A more holistic Report Builder. This allows users to define what content shows up in the report (#348).

  • Users can now compose a long form “Package Summary” to keep track of more pertinent items (perhaps items less central to {riskmetric} output) for a more rounded package review (#348).

Latest: 'Database View' Face Lift

  • More useful content

    • Date uploaded
    • Decision-related columns like the decision time stamp & decision source

  • Introduced the decision category table

Latest: Set custom org-level settings

Customizable either in-app or via the config file. What are the changes?

  • Added decision categories
    • Added decision automation rules
    • Added log file designation
    • Implemented user roles/privileges to allow for more granular access to actions in the application
    • Allow metric weights to be initialized
    • Custom colors

Latest: Edit Roles & Privileges

Latest: Explore Package Source Code

  • More manual package review process

Coming soon

{riskscore}

In it’s early stages, a new data package containing the {riskmetric} assessments & scores for all packages on CRAN.

Coming soon

Package Explorer

A suite of hands-on review tools (from our friends at GSK) will allow users to browse unit tests, source code, and man (doc) pages for any exported function.

Dev Team

Q & A